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ABSTRACT 

In the 1995-96 school year, the Memphis (Tennessee) 
City Schools released standards for student performance in seven 
content areas and began laying the foundation for a s tandards~based 
curriculum and assessment system. The steps taken to develop and 
implement this project are outlined as follows: (1) defining the 

objectives and the project scope; (2) creating a budget and funding 
plan; (3) designing the tasks; (4) explaining the project to schools 
involved; (5) administering the tasks; (6) preparing student work for 
scoring; (7) reading student work and choosing anchor papers; (8) 
training teachers to score student work using a rubric and anchor 
book; (9) compiling and analyzing the data; and (10) evaluating the 
experience. The experience of the Memphis schools indicates the 
advisability of starting on a small scale, involving the right 
people, and rewarding the participating schools. The 10-step plan can 
enable a district to launch a performance assessment project that is 
comprehensive, confidential, and responsible. (SLD) 
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10 Steps to District Performance Assessment 

Lydia Abell Driscoll, Ed.D. 

Memphis City Schools 

Department of Research, Standards and Accountability 

Jerome gazed at the straw mounted on the wall poster in his cafeteria-turned- 
testing site. "What’s My Length?" said the lettering above the straw. He knew he 
wasn't allowed to touch it. He already tried holding the imaginary length between 
his outstretched hands, but it got too big before he could get his hands to his ruler. 

All of a sudden, an idea struck Jerome. He raised his right arm, matching his elbow 
to one end of the straw, and marked off the length on his arm. Now this was a 
marker that didn’t move. Jerome entered his estimate in the test booklet and went 
on to explain how he reached his answer. 

"Testing" was more interesting for Jerome on this occasion than it had ever 
been in first or second grade. As a third grader, Jerome was included in the new 
district "Performance Assessment Pilot Project," designed as a district level audit to 
begin establishing a baseline for student performance toward the Memphis City 
Schools (MCS) Content Standards. The development of standards has been part of 
the Memphis School District restructuring plan since the development of Life-Long 
Learning Standards by a community task force in 1993-94. The "Performance 
Assessment Pilot Project" describes the process used by the Memphis School District 
to conduct an audit of student performance in 10 schools through the development 
and implementation of performance-based assessments. 

Background 

During the 1995-96 school year, the MCS Office of Student Standards released 
Draft K - 12 Content Standards and Specific Expectations in seven content areas: 
arts/visual arts/ theatre /music; English/ language arts; foreign language; health/ 
wellness/ physical education; mathematics; science, and social studies. These 
standards had been developed by action teams of MCS teachers and community 
leaders. A training plan was initiated in September, 1996 to introduce all teachers 
(7,727) to the standards through a series of faculty meetings, conducted in all schools 
(161) by their principals. In conjunction with the faculty meetings, the Performance 
Assessment Pilot Project began laying the practical foundation for a standards-based 
curriculum and assessment system. The steps taken to develop and implement this 
project are detailed below. 

Step 1: Definition of the objectives and scope for the project. 

The project had several objectives. One was to develop performance-based 
assessment tasks that were clearly linked to the content standards. Another 
objective was the implementation of the tasks in ten schools, thereby establishing a 
performance baseline for future comparison. A third objective was the training of 
teachers in the characteristics of performance assessment tasks, as well as in 
techniques for using rubrics to score student work. 
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The parameters were established for the project, to include tasks in the four 
core content areas (math, science, social studies, English /language arts), and a range 
of grade levels representative of the whole district. Of the ten schools, two were 
high schools, two were middle schools and six were elementary schools. The ten 
schools invited to participate were the same ten that had been part of a Goals 2000 
project the previous year. Because of the district's commitment to science and 
mathematics education, the ten performance assessment tasks included three in 
science and three in math, as well as two in English /language arts and two in social 
studies. It was determined that the tasks would be administered to 50 students from 
each of the ten schools. 

Step 2: Creation of a budget and funding plan. 

The budget for the MCS project included these items: stipends to pay teachers 
for the two-day scoring event; printing costs for the testing materials and anchor 
books; manipulatives used in the task administrations. 

Step 3: Design of the tasks. 

Office of Student Standards facilitators were assigned grade levels and schools 
for their task development. Some staff members designed original tasks, while 
others adapted public-domain tasks to the MCS standards. Three Check Points were 
set to provide feedback to the developers, and to guarantee that all materials would 
be ready on time for task administration. 

Step 4: Explanation of the project to schools in the process. 

Careful explanation and cooperation with the pilot schools are critical to the 
success of the project. The principals of the selected schools were invited to a 
meeting, during which the project was explained and they were given a range of 
days and times from which to choose the task administration date. Principals 
returned their Choice Form, and received a confirmation letter. The individual 
staff members who would administer the tasks visited the schools prior to the task 
administration, in order to view the space and confer with the classroom teachers 
whose children would be involved. Schools submitted lists of the students they 
expected to be involved, which provided an initial database for accurate tracking. 
These lists included the student identification numbers, which provide access to the 
district's central database. 

Step 5: Administion of the tasks. 

Office of Student Standards staff were divided into teams for task 
administration. A list of specific jobs for the Task Administrator, the Assistant 
Administrator and the Videographer was prepared. The staff held a rehearsal so all 
staff would be familiar with each task. Each task was videotaped for archive and 
training purposes. The Assistant Administrator completed the Task 
Administration Report, which provided the final list of students who participated 
in the assessment. At the end of each of the four testing days, the full staff met for a 
debriefing session, with the discussion guided by the Performance Task 
Administration Evaluation form. When all testing was completed, the student 
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work and the Task Administration Report were turned in to the project coordinator. 
All manipulates used for the project were given to the participating schools. A 
total of 446 tasks were administered in the 10 schools. 

Step 6: Preparation of the student work for scoring. 

The clerical work in this step was critical to tracking the assessment results 
and maintaining student confidentiality. A final list of student participants from 
each school was created, verified from the student work, the initial lists and the 
Task Administration Report, and sent to the MCS Research Office for the 
generation of a set of four labels with the student identification number for each 
student. Color-coded scoring sheets were printed according to content area: 

English /language arts tasks were scored with pink, turquoise and gold sheets; social 
studies tasks were scored with yellow, buff and lime sheets; science tasks were 
scored with green, violet and salmon sheets; and math tasks were scored with blue, 
tan and orange sheets. 

Clerical staff prepared the student work and three scoring sheets for each 
student by placing a student identification number label on the work and each of the 
three scoring sheets. Staff removed and collected the front covers of each test packet, 
which included the student's name, grade, race, gender and school. Each piece of 
student work was numbered by schools, according to an assigned list. The work 
packets were clipped together, ready for the staff reading sessions. 

Step 7: Reading of the student work, choosing anchor papers. 

The Standards staff was divided into reading teams. After discussing the 
prompt, the student directions and the rubric, the teams read all the student work, 
beginning to categorize it according to the scoring rubric. The teams chose a 
representative student response from each scoring level to be used as the "anchor" 
paper. The staff reading teams completed the Task Summary Sheet. An anchor 
book was compiled and printed for each task, containing the scoring rubric and the 
student anchor samples. 

Step 8: Training of teachers to score student work, using a rubric and anchor book. 

The actual scoring of the student work was done by teachers over a two-day 
period. Teachers were paid for these days, which served a two-fold function. Not 
only was the scoring accomplished, but the teachers received professional 
development in evaluation. The first half-day session was devoted to training. 

After trainees were introduced to the use of a scoring rubric, they were given the 
anchor book and rubric for one of the tasks. The trainer demonstrated the 
relationship between the rubric and the anchor paper. Trainees were then given 
five samples of student work from the same task. Working in groups of four, 
trainees discussed these samples and arrived by consensus at a score for each. The 
trainer led a discussion of the correct evaluation of each of the student samples. 

The trainees worked in pairs for the next training exercise. Using a new task, 
rubric and anchor book, the teachers scored six student work samples, using the 
double score method. 
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The Double Score Method 

The double scoring method involves identifying one teacher 
as "Scorer #1" and the other as "Scorer #2." Scorer #1 reads 
and scores three of the six papers, while Scorer #2 reads and 
scores the other three. Then the scorers switch papers so both 
scorers read all six papers. It is helpful for each scorer to use a 
separate color scoring sheet. This visual identification system 
makes it clear that each sample has been scored twice. When 
both scorers have read all the samples, they may compare 
their scores and seek to reach consensus on any work they 
scored differently. 

When all trainee pairs completed the scoring, the trainer led the whole group 
in a discussion of the process. The goal of was to reach 85% interrater reliability, or 
better. That is to say, when each pair of scorers had reached agreement on five out 
of the six student sample scores, their goal was reached. The trainer surveyed the 
group to determine if at least 80% of the pairs had reached their goal of 85% 
interrater reliability. If the group had not reached this goal, they repeated this 
exercise, using an entirely different task, rubric, anchor book, and student work 
samples. 

The actual scoring sessions began when the trainer determined an acceptable 
level of interrater reliability had been reached. Once assigned a task, each pair of 
teachers proceeded to double score the student samples. Their work was monitored 
every six samples by the "Master Scorer," a role filled by Office of Student Standards 
facilitators. For example, the math facilitator who had designed the math tasks, 
served as Master Scorer for all math scoring pairs. The Master Scorer checked each 
group of six scored student work samples, filling out a Proofing /Monitor Form. The 
purpose of this check point was to provide a quality control procedure to make sure 
interrater reilability maintained the 85% agreement level, and that the scoring 
remained consistent with the rubric. In cases of unresolved disagreement, the 
Master Scorer made the decision. When the scoring teams finished their work, they 
completed the Scoring Summary Sheet and submitted it to the Master Scorer, along 
with all student work samples. 

Step 9: Compiling and analyzing the data. 

The 446 student scores were entered in the database, and a summary report 
was produced, entitled Fall Assessment, 1996: Score Results , which is the report of 
baseline performance results. This report listed the number and percentage of 
students at each school who scored each of the rubric levels on the task (score levels 
0 - 4). The data may be analyzed and presented in appropriate formats to address any 
questions of concern to the district. For example, MCS is interested in gender 
differences in math and science performance. Consequently a table was prepared, 
entitled Science and Math: Percentage of Students Scoring "Proficient" by Gender, 
which presented the percentage of males and females scoring "3" or "4" in all 
science and math tasks at the four assessment grade levels ( 3rd, 6th, 8th, and 11th 
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grade). MCS also chose to examine the percentage of students scoring proficient by 
grade level, content area and gender across all tasks. 

Step 10: Evaluation of the experience. 

MCS personnel engaged in project evaluation were able to make several 
recommendations, including the following: 

1. Start small. Unnecessary burdens on the whole system can be 
avoided by establishing manageable parameters from the 
beginning, 

2. Involve the right people. The "right people" in the MCS 
project included eight teachers-on-assignment, who were 
helpful in task design, review and administration. The scoring 
teachers were also the "right people" as they came to realize 
how valuable the experience would be for their own classroom 
use. 

3. Reward the participating schools. This can be done by 
public recognition, and by the gifting of project manipulatives 
and other materials. 



Conclusion 

The ten step plan described in this paper can enable a district to launch a 
performance assessment project that is comprehensive, confidential and 
responsible. The confidentiality of all individual students and schools was 
maintained throughout the process. All objectives of the Memphis City Schools 
project were met in a timely manner. This result, coupled with the surge of 
enthusiasm that ignited the teachers and principals involved, meets all the criteria 
for a perfect "4:" a Distinguished Performance. 
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